Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature

Interdisciplinary scientific research (IDR) extends and challenges the study of science on a number of fronts, including creating output science and engineering (S&E) indicators. This literature review began with a narrow search for quantitative measures of the output of IDR that could contribute to indicators, but the authors expanded the scope of the review as it became clear that differing definitions, assessment tools, evaluation processes, and measures all shed light on different aspects of IDR. Key among these broader aspects is (a) the importance of incorporating the concept of knowledge integration, and (b) recognizing that integration can occur within a single mind as well as among a team. Existing output measures alone cannot adequately capture this process. Among the quantitative measures considered, bibliometrics (co-authorships, co-inventors, collaborations, references, citations and co-citations) are the most developed, but leave considerable gaps in understanding of the social dynamics that lead to knowledge integration. Emerging measures in network dynamics (particularly betweenness centrality anddiversity), and entropy are promising as indicators, but their use requires sophisticated interpretations. Combinations of quantitativemeasures and qualitative assessments being appliedwithin evaluation studies appear to reveal IDR processes but carry burdens of expense, intrusion, and lack of reproducibility year-upon-year. This review is a first step toward providing a more holistic view of measuring IDR, although research and development is needed before metrics can adequately reflect the actual phenomenon of IDR. © 2010 Published by Elsevier Ltd. 1. Purpose of this literature review Increases in interdisciplinary research (IDR) have prompted a number of reports and an expanding literature on the performancemeasures, management, and evaluation of IDR. This literature review began as a response to a request from the U.S. National Science Foundation to identify quantitative output measures (Wagner, Roessner & Bobb, 2009). Deliberations led us to expand the inquiry beyond quantitative measures to be inclusive along the following lines: ∗ Corresponding author at: School of International Affairs, Pennsylvania State University, University Park, PA 16802, USA. Tel.: +1 814 865 4294; fax: +1 814 867 0405. E-mail address: [email protected] (C.S. Wagner). 1751-1577/$ – see front matter © 2010 Published by Elsevier Ltd. doi:10.1016/j.joi.2010.06.004 C.S. Wagner et al. / Journal of Informetrics 165 (2011) 14–26 15 1. Measurement of interdisciplinary research should recognize and incorporate input (consumption) and process value (creation) components as well as output (production) while factoring in short-, middle-, and long-term impacts such as delayed citation or patent citation.1 2. Interdisciplinary research involves both social and cognitive phenomena, and both these phenomena should be reflected in any measure or assessment of interdisciplinarity.2 3. Assessment of research outputs should be broadened beyond those based in bibliometrics, while also factoring in differences in granularity and dimensions of measurement and assessment.3 This article explores ideas from each of the related literatures mentioned above, and more particularly, the current and potential interaction space between them. This review also explores several promising methods for creating indicators for policymakers, research managers, evaluators, and students of the research enterprise.